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Abstract-  We  propose  a  new  computerized  system  to  accurately 
guide  laser  shots  to  the  diseased  areas  within  the  retina  based  on 
predetermined  treatment  planning.  The  proposed  system  consists 
of  a  fundus  camera  interfaced  to  a  computer  that  allows  real-time 
capturing  and  processing  of  a  sequence  of  images  at  video  frame 
rates.  The  first  image  in  the  sequence  is  used  as  a  reference  for 
manual  treatment  planning.  A  new  segmentation  technique  was 
developed  to  discern  the  blood  vessel  tree  in  which  we  extract  the 
boundaries  of  the  wide  vessels  as  well  as  the  whole  lumen  of 
smaller  ones.  The  core  of  wide  vessels  is  then  obtained  by 
correlating  the  image  with  a  1-D  Gaussian  filter  in  two 
perpendicular  directions  and  thresholding  the  result.  A  fast 
registration  technique  is  proposed  to  automatically  track  the 
motion  of  segmented  areas  within  the  subsequent  images  in  the 
sequence.  In  this  technique,  points  satisfying  certain  developed 
significance  conditions  are  chosen  as  landmarks  in  the  reference 
frame.  Using  the  extracted  set  of  corresponding  points  in  the 
subsequent  image  frames,  we  can  accurately  align  image  frames  to 
compensate  for  eye  movements  and  saccades.  The  new  technique 
has  the  potential  to  significantly  improve  the  success  rates  in  this 
type  of  treatment. 
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I.  Introduction 

Diabetic  retinopathy  resulting  from  long  term  diabetes 
mellitus  is  one  of  the  common  diseases  that  lead  to  choroidal 
neovascularization  (CNV).  CNV  is  considered  among  the 
leading  causes  of  blindness  [1].  Among  the  currently  available 
treatment  methods,  laser  can  be  used  to  photocoagulate  the 
diseased  areas.  To  obtain  satisfactory  results,  the  physician 
must  identify  the  full  extent  of  CNV  and  cauterize  it  completely 
in  order  to  save  the  central  vision  [1-2].  Several  thousand  laser 
shots  are  usually  required  during  such  treatment.  Moreover, 
special  care  must  be  taken  to  avoid  hitting  the  blood  vessel  tree, 
the  macula,  the  optic  disk,  and  the  region  between  them.  For  a 
single  eye,  this  procedure  requires  up  to  several  hours  that  are 
usually  divided  over  many  treatment  sessions  [1-2]. 

Even  though  laser  treatment  has  been  shown  to  be  superior 
to  the  other  available  methods,  it  suffers  from  a  number  of 
serious  problems.  The  current  success  rate  of  this  procedure  is 
below  50%  for  eradication  of  CNV  following  one  treatment 
session  with  a  recurrence  and/or  persistence  rate  of  about  50% 
[2].  The  latter  condition  requires  repeating  the  treatment  and 
each  treatment  repetition  in  turn  has  a  50%  failure  rate.  Several 
studies  have  indicated  that  incomplete  treatment  was  associated 
with  poorer  prognosis  than  no  treatment  at  all  [2]. 
Consequently,  the  development  of  an  accurate  laser  treatment 
guidance  system  to  treat  the  whole  retina  in  one  session  would 
have  a  strong  impact  on  the  effectiveness  of  such  procedures. 
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In  this  work,  we  propose  a  new  computerized  treatment 
planning  system  for  laser  treatment  of  CNV.  The  new  system 
has  the  potential  to  improve  the  accuracy  of  the  procedure  thus 
lowering  its  failure  rate  while  maintaining  the  same  treatment 
time.  A  major  part  of  this  system  is  the  tracking  the  retina  using 
a  computer-controlled  system  that  captures  and  processes 
retinal  images  from  a  fundus  camera.  Novel  image  processing 
routines  are  applied  to  the  retinal  images  to  determine  the 
correct  positions  for  laser  shots  on  the  retina.  In  particular,  an 
accurate  segmentation  procedure  is  applied  to  the  first  retinal 
image  to  extract  the  sensitive  areas  in  the  retina.  Subsequently, 
this  segmentation  is  continuously  updated  using  a  fast 
registration  procedure  to  obtain  the  new  positions  in  case  of  eye 
movement  at  real-time  rates.  These  positions  can  be  fed  to  a 
beam  steering  apparatus  that  precisely  directs  the  laser  beam 
accordingly.  The  new  system  is  demonstrated  on  real  data  and 
its  performance  is  analyzed. 

II.  Data  Acquisition 

A  block  diagram  of  the  data  acquisition  system  is  presented 
in  Fig.  1.  The  retinal  images  are  acquired  using  a  TOPCON 
TRC-501A  fundus  camera.  A  Sony  Charge  Coupled  Device 
(CCD)  video  camera  is  attached  to  the  eyepiece  of  the  fundus 
camera  to  make  its  images  available  in  standard  video  format. 
The  video  output  from  the  CCD  camera  is  interfaced  to  a 
Micron  PC  (700  MHz  processor,  128Mbytes  of  RAM)  through 
a  Matrox  Meteor  video  digitizer  card.  The  retinal  images  are 


Figure  1 :  a  block  diagram  for  retinal  tracking  system. 

captured  into  the  memory  of  the  computer  either  as  individual 
frames  or  as  a  real-time  video  sequence.  The  size  of  each  frame 
is  640x480  pixels  of  gray-level  images. 

III.  Methods 

A.  Contour  Detection 

Contour  detection  is  performed  using  deformable  models. 
This  technique  is  an  iterative  process  in  which  we  initialize  an 
active  contour  model  close  to  the  contour  of  interest.  This 
contour  deforms  in  such  a  way  that  it  converges  to  the  true 
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vessel  contour.  This  is  achieved  using  an  iteration  that 
minimizes  energy  functional  of  the  form: 

£  —  2  (Sintemal  (v(s))  +  £extemal(v(s))),  (1) 

where  v(s)  is  a  parameterized  contour  that  is  normally 
represented  by  a  discrete  set  of  points  or  snaxels{vh..,vn }, 
£intemai(v(s))  is  lhe  internal  energy  of  the  contour  due  to  the 
elastic  deformation  and  bending  of  the  contour,  and  £extemai(v(s)) 
represents  the  energy  due  to  image  forces  such  as  lines,  edges, 
termination  of  line  segments  and  corners  [4].  The  steady-state 
position  of  the  contour  is  the  one  it  takes  when  its  energy  is  at 
minimum.  The  contour  model  is  deformed  using  one  of  many 
algorithms  for  searching  the  neighboring  points  of  each  snaxel 
to  extract  the  point  of  minimum  energy,  which  will  then 
become  the  new  snaxel  [4]. 


Figure  2:  Initial  contour  (left)  and  after  50  iterations  (right). 


In  applying  deformable  models  to  retinal  images  to  detect 
the  boundaries  of  the  blood  vessel  tree,  the  initial  contour  has  to 
grow  and  converge  to  the  edges  of  the  vessel.  The  contour 
growth  could  be  achieved  by  interpolating  the  points  on  the 
contour.  Also,  in  order  to  obtain  useful  results,  many 
constraints  and  user  interaction  were  needed  which  limit  the 
practicality  of  the  procedure.  Here,  we  tried  to  minimize  the 
number  of  required  computations  and  make  the  process  as  close 
to  being  automatic  as  possible  at  the  highest  attainable 
accuracy.  In  particular,  we  studied  the  effect  of  the  internal 
energies  and  compared  it  with  the  effect  of  the  external  energy 
(edge  energy  in  this  case)  to  make  sure  that  the  effect  of  the 
external  energy  will  be  dominant.  The  image  was  convolved  by 
a  Sobel  operator  in  both  the  horizontal  and  vertical  directions 
[4].  The  magnitude  of  the  result  was  used  to  represent  the 
external  energy  of  the  contour.  Instead  of  selecting  a  single 
point  having  the  minimum  energy  (i.e.,  maximum  edge  value) 
from  the  neighborhood  of  each  snaxel,  we  search  all  the 
neighboring  points  of  each  snaxel  and  extract  all  the  points 
having  a  value  above  certain  threshold  as  new  points  on  the 
contour.  This  not  only  deforms  the  contour  but  also  makes  it 
grow. 


Figure  3:  Seed  contours  (left)  and  segmented  blood  vessel  tree  (right). 


B.  Detection  of  the  blood  vessel  tree 

Instead  of  manual  initialization  of  contours,  the  whole 
image  was  covered  automatically  with  an  initial  set  of  small 
contours  called  seed  contours  (see  left  image  in  Fig.  2)  [5]. 
During  the  contour  iteration  convergence  and  growth,  the  seed 
contours  that  lie  within  the  areas  of  poor  edges  shrink  until 
vanishing.  The  others  will  merge  and/or  split  until  recovering  a 
continuous  description  of  all  edges  that  pass  above  a  certain 
threshold.  This  threshold  changes  for  each  iteration  to  allow  for 
faster  and  more  accurate  convergence.  This  process  is  repeated 
until  no  changes  occur  during  a  given  iteration.  This  process 
leads  to  the  complete  detection  of  all  small  vessels  in  addition 
to  the  boundaries  of  the  larger  ones.  The  intensity  values  across 
blood  vessels  are  usually  assumed  to  have  a  Gaussian 
distribution.  On  the  other  hand,  the  small  segments  usually 
have  repeated  intensity  values.  So,  correlating  the  image  with 
two-dimensional  filters  of  suitable  size  (e.g.,  16x15)  can 
provides  reasonable  results  but  requires  a  large  number  of 
computations  to  detect  all  vessels.  In  particular,  the  filter  has  to 
be  applied  in  12  different  directions  to  detect  vessels  in  all 
possible  directions.  Since  we  detect  the  small  vessels  and  the 
boundaries  of  the  wider  vessels  at  an  earlier  step,  we  only  need 
to  obtain  the  core  of  the  wide  vessels.  Therefore,  we  only  have 
to  correlate  the  image  with  two  one-dimensional  Gaussian  filter 


Figure  4:  Examples  of  demarcation  of  the  optic  disc  and  the  macula. 


in  the  vertical  and  the  horizontal  directions  only.  This  reduces 
the  computational  complexity  of  the  original  technique  to  a 
great  extent  and  makes  it  practical  for  clinical  applications.  Fig. 
2  shows  the  seed  contours  and  the  result  of  applying  this 
technique. 

C.  Segmenting  the  optic  disc  and  the  macula 

The  optic  disc  is  a  well-delineated  object  with  properly 
defined  size  and  edges.  In  the  gray  level  retinal  images,  it 
always  appears  as  nearly  circular  structure  that  is  brighter  than 
the  background  of  the  retina.  So,  the  Hough  transform  [7], 
which  is  well  known  robust  method  to  extract  circular  objects, 
was  used  to  detect  the  center  and  the  radius  of  the  optic  disc.  A 
circle  was  drawn  around  the  optic  disc  and  this  circle  was  filled 
to  distinguish  this  structure. 

The  macula  is  the  area  of  acute  vision  within  the  retina.  It 
appears  as  the  most  homogeneous  area  near  the  optic  disc.  To 
segment  the  macula,  we  used  a  region-growing  algorithm  to 
demarcate  the  macula  [7].  Fig.  3  shows  the  result  of  applying 
these  techniques  on  retinal  images  with  Indocyanine  Green 


contrast  (which  appears  as  bright  blood  vessels  in  a  dark 
background)  as  well  as  normal  images. 

D.  Estimating  the  locations  of  laser  shots 

After  extracting  all  sensitive  objects  that  must  be  avoided 
during  laser  treatment,  a  binary  image  was  composed 
containing  these  objects.  This  image  was  dilated  by  a  square 
structuring  element  of  dimensions  7x7  to  maintain  a  safety 
margin  around  these  sensitive  areas.  The  locations  of  shots 
were  determined  and  spaced  in  the  background  (see  Fig.  4). 
These  locations  are  stored  in  the  database  of  the  patient  and  get 
updated  with  every  successive  image  frame  [7]. 

E.  Real-time  movement  tracking 

Since  the  segmentation  step  is  rather  time  consuming,  it  is 
not  possible  to  use  it  to  track  fast  eye  movements  particularly 
during  the  laser  treatment  session.  Alternatively,  we  only  use 
segmentation  for  the  first  (reference)  retinal  image  and  then  the 
segmentation  of  any  subsequent  frame  is  obtained  through  fast 
registration  with  this  reference.  Since  the  retinal  motion  is 
expected  to  follow  a  rigid  body  motion  model  (except  in  case  of 
retinal  tears),  we  developed  an  algorithm  to  extract  some  points 
(landmarks)  and  get  their  relation  relative  to  certain  point.  Then 
we  find  their  corresponding  points  in  the  subsequent  frames. 

Algorithm  for  detection  of  candidate  landmarks 

1 .  Start  with  a  reference  image  R  ,  a  current  image  C,  a 
threshold  t  and  a  one-dimensional  filter  /  defined  as 
[4, 3, 2, 1,-2, -5, -6, -5, -2, 1,2, 3, 4]. 

2.  Estimate  RefC ,  the  optic  disc  center,  using  Hough 
transform. 

3.  Correlate  R  with  /  and  threshold  above  t.  let  the  result  be 
called  RR. 

4.  Perform  morphological  thinning  of  RR  by  one  pixel. 

5.  Let  temp  =  sum  of  a  5x5  window  over  every  pixel  in  RR. 

6.  Let  RefLandmarks  be  the  points  in  temp  that  are  greater 
than  5  (points  of  branching  and  cross  over).  We  get  n 
points. 

7.  If  there  are  some  connected  points,  take  only  the  central 
point.  Hence,  the  number  of  points  reduces  to  m  points. 

8.  Take  a  window  around  the  optic  disc  center  and  get  the 
corresponding  image  called  CurrentC  in  the  subsequent 
frames  by  template  matching. 

9.  Repeat  steps  3-7  for  the  current  image  C  to  obtain  the 
CurrentLandmarks  as  m  points. 

Then,  we  have  a  set  of  reference  landmarks  and  a  set  of  current 
landmarks. 

Algorithm  to  obtain  the  corresponding  pairs 

1.  Take  RefC  as  the  point  of  origin  and  compose  m  vectors  for 
the  m  points  in  CurrentLandmarks. 

2.  Compute  the  slope  of  every  pair. 

3.  Repeat  steps  1-2  for  CurrentC  to  obtain  n  vectors.  Every 
pair  in  the  reference  set  may  have  a  corresponding  pair  in 
the  current  set. 

4.  Calculate  the  angle  of  rotation  for  every  two  pairs. 


5.  Extract  the  pairs  of  angles  with  absolute  angle  magnitudes 
between  4  and  6  or  less  than  1 . 

6.  Transform  each  of  the  points  in  RefLandmarks  by  an  affine 
transformation  matrix  corresponding  to  the  extracted 
rotation  angle. 

7.  Find  points  in  CurrentLandmarks  that  are  close  to  the 
result  of  transformation.  Then  we  have  a  set  of  reference 
points  and  their  corresponding  points  in  the  current  image. 

8.  Take  the  mean  angle  of  rotation  and  the  mean  shifts  to  be 
the  motion  parameters. 

9.  Transform  the  position  of  laser  shots  in  the  reference  frame 
to  their  corresponding  positions  in  the  current  frame  using 
the  computed  motion  model. 

It  should  be  noted  here  that  one  could  use  the  vector  length 
instead  of  the  transformation  matrix  for  a  faster  response  at  still 
reasonable  results. 

IV.  Results  and  Discussion 

The  results  of  segmenting  the  blood  vessel  tree  are  shown  in 
Fig.  3.  As  can  be  observed,  the  proposed  algorithm  gives 
accurate  results.  The  grid  of  seed  contours  makes  the  process 
close  to  being  automatic.  Fig.  4  shows  an  example  of 
demarcation  of  the  optic  disc  and  the  macula.  After  extracting 
all  of  the  sensitive  areas  in  the  reference  frame,  we  locate  the 
laser  shots  away  from  these  objects.  The  center  of  the  optic  disc 
was  shown  to  be  an  excellent  landmark  between  different 
image  in  the  acquired  image  sequence.  This  is  mainly  because 
the  gray  scale  pattern  around  this  center  cannot  be  matched 
anywhere  else  within  the  image.  This  is  important  to  make  the 
registration  process  more  robust. 

The  method  suggested  for  landmark  estimations  used  the 
same  criteria  in  all  image  frames.  So,  we  expect  it  to  give  fairly 
accurate  list  of  candidate  landmarks  for  subsequent  matching  in 
our  registration  procedure.  Fig.  5  shows  an  example  of  the 
obtained  landmarks  in  the  reference  and  another  image  in  the 
sequence  of  images  following  the  reference  image.  As  can  be 
seen,  most  of  these  landmarks  have  correct  match  in  the  other 
image,  which  indicates  the  success  of  the  proposed  matching 
procedure.  Fig.  6  shows  two  example  of  mapping  the  laser 
shots  in  normal  images  (top  pair)  as  well  as  Indocyanine  Green 
images  (bottom  pair).  The  results  show  an  accurate  mapping  of 
laser  shot  locations  between  the  reference  (left)  images  and  the 
other  image  on  the  right.  This  demonstrates  the  success  of  the 
proposed  technique  in  tracking  the  motion  of  the  retina  and 
correcting  the  target  laser  shot  locations  accordingly. 

It  is  important  to  comment  on  the  computational  complexity 
of  the  proposed  algorithm.  The  first  part  of  the  technique 
involving  the  segmentation  of  the  blood  vessel  tree  from  the 
reference  image  has  the  highest  computational  complexity  in 
the  new  method.  The  exact  time  required  for  doing  this  task 
varies  from  image  to  another.  Nevertheless,  the  observed 
typical  time  required  for  such  procedure  was  between  4-7  sec 
on  the  data  acquisition  computer  system  described  above.  This 
time  is  not  crucial  for  our  real-time  processing  goal  since  this 
processing  is  done  prior  to  the  actual  laser  treatment.  Therefore, 
the  time  to  extract  and  match  the  landmarks  to  register  other 


images  to  that  reference  image  is  the  most  important  for  the 
practicality  of  the  proposed  method.  The  computations  of  this 
part  may  also  vary  between  different  images  in  the  sequence. 
The  computational  times  we  measured  for  our  experiments 
provided  results  in  the  range  between  0.2  and  0.3  sec,  which  is 
considered  practical  for  on-line  use.  Using  a  more  powerful 
computing  platform  (which  is  now  widely  available 
commercially)  can  bring  this  time  lower. 

Even  though  the  proposed  technique  shows  a  large  potential 
for  clinical  application,  several  limitations  are  yet  to  be 
addressed  for  more  robust  performance.  Some  pathological 
cases  involve  dramatically  different  retinal  image  morphology 
that  would  break  the  assumptions  used  in  this  work.  An 
example  of  this  is  the  case  where  noncircular  optic  disk. 
Moreover,  the  proposed  technique  cannot  handle  large  scale, 
contrast  or  illumination  changes  among  images  within  the 
acquired  image  sequence.  Such  changes  may  result  from 
normal  effects  like  fading  contrast  agent  or  gross  motion.  It  is 
therefore  recommended  that  the  reference  image  be  updated 
periodically  during  the  laser  treatment  session.  This  helps  keep 
the  acquired  images  always  within  an  acceptable  range  from  the 
reference  image.  Another  limitation  in  computing  the  rotational 
motion  component  is  the  limited  resolution  of  images.  This 
result  in  approximating  fractional  laser  shot  locations  into  their 
nearest  integer  coordinates.  As  a  result,  the  accuracy  of  the 
derived  laser  shot  locations  cannot  get  any  better  than  half  a 
pixel. 

V.  Conclusions 

A  new  computerized  technique  for  laser  treatment  planning 
was  proposed.  The  new  technique  performs  accurate 
segmentation  on  a  reference  frame.  The  locations  of  the  laser 
shot  on  this  reference  image  are  then  recomputed  for  the 
sequence  of  images  acquired  during  the  treatment  using  fast 
registration.  This  enables  fast  tracking  of  retinal  structures  and 
ensures  proper  administration  of  the  treatment  in  case  of  eye 
movement.  The  proposed  technique  was  applied  to  real  data 
and  the  results  showed  a  large  potential  for  its  clinical 
application. 
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Figure  5:  Landmarks  in  reference  image  (left)  and  a  subsequent  image  (right). 


Figure  6:  The  reference  location  of  laser  shots  (left  images)  and  the 
transformed  location  (right  images). 


